Gibbs Sampling in Open-Universe Stochastic Languages
نویسندگان
چکیده
Languages for open-universe probabilistic models (OUPMs) can represent situations with an unknown number of objects and identity uncertainty. While such cases arise in a wide range of important real-world applications, existing general purpose inference methods for OUPMs are far less efficient than those available for more restricted languages and model classes. This paper goes some way to remedying this deficit by introducing, and proving correct, a generalization of Gibbs sampling to partial worlds with possibly varying model structure. Our approach draws on and extends previous generic OUPM inference methods, as well as auxiliary variable samplers for nonparametric mixture models. It has been implemented for BLOG, a well-known OUPM language. Combined with compile-time optimizations, the resulting algorithm yields very substantial speedups over existing methods on several test cases, and substantially improves the practicality of OUPM languages generally.
منابع مشابه
Parsing low-resource languages using Gibbs sampling for PCFGs with latent annotations
PCFGs with latent annotations have been shown to be a very effective model for phrase structure parsing. We present a Bayesian model and algorithms based on a Gibbs sampler for parsing with a grammar with latent annotations. For PCFG-LA, we present an additional Gibbs sampler algorithm to learn annotations from training data, which are parse trees with coarse (unannotated) symbols. We show that...
متن کاملStochastic Attribute-Value Grammars
Probabilistic analogues of regular and context-free grammars are wellknown in computational linguistics, and currently the subject of intensive research. To date, however, no satisfactory probabilistic analogue of attribute-value grammars has been proposed: previous attempts have failed to define a correct parameter-estimation algorithm. In the present paper, I define stochastic attribute-value...
متن کاملGibbs Sampling, Exponential Families and Coupling
We give examples of a quantitative analysis of the bivariate Gibbs sampler using coupling arguments. The examples involve standard statistical models – exponential families with conjugate priors or location families with natural priors. Our main approach uses a single eigenfunction (always explicitly available in the examples in question) and stochastic monotonicity.
متن کاملGibbs Sampling for Signal Reconstruction
This paper describes the use of stochastic simulation techniques to reconstruct biomedical signals not directly measurable. In particular, a deconvolution problem with an uncertain clearance parameter is considered. The problem is addressed using a Monte Carlo Markov Chain method, called the Gibbs Sampling, in which the joint posterior probability distribution of the stochastic parameters is es...
متن کاملLocal Expectation Gradients for Black Box Variational Inference
We introduce local expectation gradients which is a general purpose stochastic variational inference algorithm for constructing stochastic gradients by sampling from the variational distribution. This algorithm divides the problem of estimating the stochastic gradients over multiple variational parameters into smaller sub-tasks so that each sub-task explores intelligently the most relevant part...
متن کامل